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ABSTRACT 


^  This  is  an  analysis  of  existing  feedback  in  the  Fleet 
Modernization  Program  ( FMP )  planning  and  design  process 
using  the  FFG-7  class  of  ships  as  a  case  study.  This 
analysis  attempts  to  relate  the  engineering  drawing  revision 
rate  (inverse  measure  of  drawing  quality)  and  the  number  of 
ships  affected  by  those  drawing  revisions  (measure  of 
availability  concurrence)  to  the  cost  growth  attributable  to 
the  FMP  portion  of  U.S.  Navy  ship  availabilities.  Due  to 
the  lack  of  actual  cost  data,  budget  estimates  were  used  as 
a  surrogate  and  unfortunately  firm  relationships  could  not 
be  established. 

However,  the  methodology  developed  has  potential  for 
application  to  any  large  ship  class  which  may  experience 
numerous  concurrent  availabilities,  as  actual  cost  data 
become  available.  It  is  meant  to  be  a  tool  for  the 
engineering  design  agent  to  assess  the  financial  impact  of 
the  quality  of  engineering  design  products  on  the  installing 


activities  and  to  assess  the  potential  value  of  policy 
changes  which  improve  the  quality  of  those  products.  • 
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Improving  the  quality  and  efficiency  of  the  process  through 
innovative,  cost-effective  policy  alternatives. 

Once  ships  are  built,  they  leave  the  protective  umbrella 
of  the  procurement  world  and  enter  the  realm  of  maintenance 
and  upkeep.  The  maintenance  and  upkeep  functions  can  be 
decomposed  into  two  areas  of  importance:  1)  repair  -  fixing 
existing  shipboard  systems  and  2)  upgrade  -  removal  of 
obsolete  systems,  improvement  of  existing  systems,  or 
installation  of  new  systems.  Repair  planning  and 

accomplishment  is  the  responsibility  of  the  ship's  force, 
the  Planning,  Estimating,  and  Repair  Activity  (PERA),  and 
the  Type  Commander  (TYCOM).  Upgrade  is  the  responsibility 
of  the  Commander  Naval  Sea  Systems  Command  (NAVSEA)  which  is 
accomplished  through  the  FMP .  The  scope  of  this  study  is 
strictly  concerned  with  the  upgrade  planning  and  design  as 
accomplished  through  the  FMP,  specifically  using  the  FFG-7 
class  as  a  case  study. 

A.  BACKGROUND 

The  planning,  design,  and  installation  responsibilities 
for  implementation  of  the  FMP  are  spread  among  many 
subordinate  commands  which  must  be  properly  sequenced  and 
coordinated  to  assure  timely  and  efficient  upgrade  of 
specific  ships.  This  section  describes  the  principal 
players  and  their  responsibilities.  For  easy  reference,  a 
glossary  has  been  provided  in  Appendix  A. 


The  Ship's  Logistic  Manager  ( SLM ) ,  located  at  NAVSEA 
headquarters  in  Washington,  D.C.,  in  conjunction  with  his 
counterpart  in  the  office  of  the  Chief  of  Naval  Operations, 
programs  specific  ship  alterations  (SHIPALT's)  for 
accomplishment  on  individual  ships  during  their  respective 
availabilities.  Additionally,  the  SLM  tasks  the  Expanded 
Planning  Yard  ( EPY ) ,  responsible  for  his  ship  class,  with 
SHIPALT  development  for  future  accomplishment. 

The  EPY,  as  the  design  agent,  is  responsible  for  SHIPALT 
development  for  a  given  ship  class.  He  produces  all 
installation  drawings  for  each  SHIPALT  and  tailors  those 
drawings  to  each  specific  ship.  A  list  of  applicable 
drawings  along  with  the  drawings  themselves  are  then 
provided  to  the  activities  responsible  for  installation  of 
the  SHIPALT's  on  each  ship  to  facilitate  writing  and  award 
of  contracts. 

In  the  case  of  public  sector  availabilities,  i.e.,  those 
accomplished  by  Naval  Shipyards,  the  responsible  Supervisor 
of  Shipbuilding  and  Ship  Repair  (SUPSHIP)  merely  issues  the 
work  specifications  to  the  installing  shipyard  and  the 
funding  for  the  work  comes  directly  from  NAVSEA  to  the 
shipyard.  The  other  case  is  that  of  availabilities  serviced 
by  the  private  sector,  in  which  the  SUPSHIP  issues  a 
contract  to  a  private  shipyard  and  the  funding  comes  from 
NAVSEA  via  the  responsible  SUPSHIP,  who  is  the  contracting 
agent,  to  the  Installing  shipyard. 


For  either  case,  the  specifications  for  improvements  to 
a  ship  are  based  on,  and  referenced  to,  the  drawing  schedule 
(list  of  applicable  drawings)  provided  by  the  EPY  to  the 
responsible  SUPSHIP  approximately  twelve  months  prior  to 
the  ship's  availability  start  date  (Ref.  1).  In  the  past, 
when  the  number  of  ships  in  a  class  was  relatively  small 
(less  than  10  or  15)  and  the  maintenance  philosophy  was 
centered  around  an  intensive  overhaul  every  five  years,  this 
lead  time  presented  no  problem  since  availabilities  rarely 
overlapped.  No  overlap  meant  lessons  learned  on  one 
availability  could  be  incorporated  easily  into  the  next 
subsequent  contract.  Currently,  with  the  existence  of 
several  large  ship  classes  (FFG-7,  DD-963,  etc.)  of  more 
than  20  ships  each,  accompanied  by  a  phased  maintenance 
philosophy  which  calls  for  short  availabilities  every  tvo 
years,  concurrent  availabilities  are  unavoidable.  Thir 
creates  the  situation  in  which  a  poor  drawing  encountered  on 
one  installation  can  affect  several  other  concurrent 
Installations  as  well  as  any  upcoming  availabilities  in 
which  the  contract  is  already  awarded.  This  effect  will  be 
referred  to  throughout  this  study  as  the  "snowball"  effect. 
To  resolve  drawing  deficiencies,  the  basic  drawings  must  be 
revised  and  the  drawing  schedules  updated  to  reflect  the 
revised  drawings.  Obviously,  as  the  revision  rate 
increases,  the  drawing  schedules  will  become  obsolete  much 
more  rapidly. 
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Review  of  contract  completion  reports  for  FFG-7  class 
availabilities  during  1985,  1986,  and  1987  revealed  two 
contributors  to  contract  cost  growth  on  FMP  items.  The 
first,  incomplete  specifications  which  do  not  reference  the 
proper  drawings  (due  to  an  obsolete  drawing  schedule), 
manifests  itself  as  a  costly  contract  modification  early  in 
the  availability  to  incorporate  the  correct  drawings.  The 
second,  drawing  deficiencies  encountered  during 
installation,  manifest  themselves  as  contract  modifications 
throughout  the  availability  period.  In  addition,  associated 
government  delay  and  disruption  charges  may  be  a  source  of 
cost  growth  which  lingers  long  after  availability  completion 
during  potential  litigation. 

Drawing  deficiencies,  hypothesized  as  a  variable 
directly  associated  with  cost  growth  during  FMP 
implementation,  are  the  root  of  the  problem  addressed  in 
this  study.  The  basic  problem  is  that  existing  feedback  to 
the  EPY  in  the  form  of  Liaison  Action  Records  (LAR's)  is 
inadequate  for  the  EPY  to  assess  the  Impact  of  its  product 
quality  on  the  installing  activities. 

To  avoid  any  additional  accounting  or  administrative 
burdens  on  the  installing  activities,  this  study  attempts  to 
relate  installation  cost  growth  to  the  LAR  as  the  indicator 
of  EPY  product  quality,  in  this  case  drawings  and  drawing 
schedules.  Hopefully,  this  will  in  turn  enable  the  EPY  to 
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conduct  an  educated  cost-benefit  analysis  when  considering 
options  for  improving  the  quality  of  their  products. 

B.  ORGANIZATION 

The  next  chapter  explains  the  steps  taken  to  obtain  a 
viable  data  base  for  establishing  the  relationship  between 
design  deficiencies  and  cost  growth.  It  will  also  cover 
definitions  of  the  key  variables  and  all  adjustments  made  to 
the  raw  data  to  make  it  commensurable. 

The  third  chapter  covers  the  model  development.  The 
fourth  chapter  includes  results  and  conclusions  about  the 
model  and  its  shortcomings  along  with  significant  problem 
insights  obtained  during  model  development.  Chapter  five 
consists  of  recommendations  for  further  study.  All  raw  data 
and  adjusted  data  are  appended  at  the  end  of  the  study. 


P 
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£ 
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A.  SELECTION  PROCESS 

Once  the  problem  was  defined  as  discussed,  the  crucial 
issue  became  one  of  finding  a  suitable  data  base  from  which 
to  derive  the  desired  relationship.  The  search  for  a  data 
base  appeared  to  be  easy  at  first  glance,  but  required  three 
iterations,  with  increasing  numbers  of  assumptions  at  each 
stage.  This  chapter  outlines  the  data  desired,  definitions, 
and  the  actual  data  obtained  for  the  study. 


The  first  "rough  cut"  at  tying  increased 
installation  costs  to  EPY  drawing  revisions  was  attempted  at 
a  "micro"  level.  Specific  Liaison  Action  Records  and  their 
associated  drawing  revisions  for  the  FFG-10's  availability 
conducted  in  Long  Beach,  CA  between  September  and  December 
1985  were  recorded  and  the  resultant  list  was  taken  to 
SUPSHIP  Long  Beach,  CA  who  contracted  the  availability. 
Using  the  Contract  Completion  File,  an  attempt  was  made  to 
account  for  the  cost  growth  by  associating  a  contract 
modification  with  each  LAR. 

This  approach  proved  to  be  infeasible  for  two 
reasons.  The  first  is  that  contract  completion  files  are 
not  centrally  located,  i.e.,  each  SUPSHIP  maintains  them  at 
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their  location.  Additionally,  the  files  are  hard  copy  only. 


which  requires  an  investment  in  time  to  learn  the 
idiosyncracies  of  each  SUPSHIP's  filing  system.  Had  the 
resources  been  available  to  visit  each  cognizant  SUPSHIP  and 
review  their  files,  this  data  could  have  been  collected,  but 
the  research  would  not  have  been  productive  for  the  second 
reason. 

The  second  problem  with  this  approach,  the  fixed  price 
contracting  policy  of  the  Navy  during  the  period  being 
studied,  made  it  virtually  impossible  to  associate  a 
contract  modification  with  any  one  item.  Each  contract 
modification  was  a  lump  sum  increase/decrease  to  the 
contract  award  price  arrived  at  through  negotiations  with 
the  contractor  for  a  group  of  items.  For  the  contract 
reviewed,  only  about  38%  of  the  LAR's  on  the  list  were 
eventually  recorded  as  contract  modifications. 

2.  Concept  Modification  Number  1 

To  avoid  the  complications  associated  with  a  "micro" 
approach,  the  next  attempt  was  to  look  at  installation  cost 
growth  as  the  difference  between  the  final  contract  cost  and 
the  initial  contract  award  price.  This  approach  too,  had 
complications  which  made  it  unworkable  for  this  study.  In 
the  future,  the  complications  can  be  overcome  making  this 
the  most  promising  approach  for  future  studies. 

Due  to  fixed  price  contracting,  it  is  not  possible 
to  directly  separate  cost  growth  in  the  repair  package  from 


cost  growth  in  the  FMP  package.  This  can  be  overcome 
through  SUPSHIP  estimates  of  NAVSEA's  share  of  the  award 
price  and  the  completion  price.  The  difference  between 
these  two  figures  would  represent  the  growth  attributable  to 
NAVSEA.  Additionally,  by  referencing  the  data  base  to  be 
described  in  the  next  section,  any  growth  due  to  changes  in 
contract  scope1  can  be  taken  into  account. 

The  other  complication  which  presently  makes  this 
option  undesirable  is  that  the  majority  of  SUPSHIP's 
involved  with  FFG-7  work  did  not  come  on  line  with  the 
Navy's  automated  accounting  system  (STAR)  until  mid-  to 
late-1986.  This  leads  to  the  same  complication  as  the 
initial  concept  —  scattered,  hardcopy  records.  To  obtain  a 
complete  data  base  for  the  period  of  the  study  (1985  -  May 
1987)  would  require  visiting  individual  SUPSHIP's  to  obtain 
contract  completion  data  for  availabilities  they 
administered  prior  to  coming  on-line  with  STAR.  Worth 
noting  here,  by  the  end  of  FY88,  enough  complete  data  will 
be  in  the  STAR  data  base  (a  long  enough  period  of 
expenditure  information  to  be  of  use)  to  make  this  a  viable 
approach. 

3.  Concept  Modification  Number  2 

Since  actual  cost  data  were  not  readily  available. 


x.  Changes  in  scope  are  modifications  which  add  new 
work  or  delete  existing  work  in  the  contract  as  opposed  to 
changes  within  scope  which  are  modifications  to  procedures 
or  drawings  pertaining  to  existing  work  in  the  contract. 
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an  alternative  data  base  of  funding  and  fiscal  program  data 
for  the  FMP  ( SAFI RE)  was  explored.  This  data  base  contains 
only  FMP  information  so  the  previous  problem  of  separating 
repair  and  modernization  costs  does  not  exist. 
Additionally,  changes  in  availability  cost  due  to  changes  in 
scope  can  be  accounted  for  by  looking  at  the  current  amount 
programmed  through  the  escrow  account  for  each  hull. 

To  define  availability  cost  growth,  the  following 
assumptions  were  made:  1)  the  current  amount  programmed  for 
a  given  ship  is  an  accurate  and  consistent  estimate  of  its 
actual  availability  end  cost  and  2)  the  amount  funded  to  the 
contracting  SUPSHIP  is  approximately  the  contract  award 
price.  With  these  two  assumptions,  growth  was  defined  as 
the  difference  between  the  amount  programmed  and  the  amount 
funded.  This  crude  definition  of  growth  actually  reflects 
the  accuracy  of  NAVSBA's  budget  estimates  rather  than  any 
actual  cost  growth,  but  will  be  used  as  a  surrogate  measure 
of  actual  cost  growth. 

B.  VARIABLE  DEFINITIONS 

1 .  Response  Variable  -  Cost  Growth 

Despite  the  foregoing  discussion,  growth  by  hull  was 
defined  as  the  difference  between  the  current  amount 
programmed  and  the  total  amount  funded  as  documented  in  the 
SAFIRE  (budget  execution  subsystem)  data  base  (see  Appendix 
B).  To  develop  the  time  series  data  for  the  model,  the 
growth  for  a  given  hull  was  first  adjusted  to  constant  Bast 
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Coast,  1986  dollars  (see  Appendix  C) ,  and  then  averaged  over 
the  number  o£  days  the  ship  was  In  an  availability  to  arrive 
at  an  average  daily  cost  growth  figure  for  each  hull.  The 
temporal  and  regional  adjustments  to  the  data  base  will  be 
discussed  below.  The  average  dally  growth  figure  was  then 
multiplied  by  the  number  of  days  the  ship  spent  in  its 
availability  each  month  to  arrive  at  an  average  growth 
figure  for  each  month  of  the  availability.  These  monthly 
growth  figures  were  then  summed  over  all  ships  in  an 
availability  for  a  given  month  (see  Appendix  D).  The  result 
was  29  periods  of  average  monthly  cost  growth  for  the  FFG-7 
class  between  January  1985  and  May  1987.  Specifically  left 
out  of  the  data  base  were  the  FFG-8  and  the  FFG-16.  Their 
respective  availabilities  were  unique  and  of  extended 
duration. 

2.  Explanatory  Variables 
a.  Revision  Rate 

As  discussed  in  the  introduction/  the  revision 
rate  experienced  by  the  EPT  is  the  only  consistent  feedback 
they  receive  concerning  their  products  —  drawings.  This 
was  chosen  as  the  measure  of  SPY  drawing  quality. 
Theoretically/  a  high  revision  rate  at  the  EPY  should  be 
reflected  by  a  large  amount  of  cost  growth  during 
installation.  To  quantify  revision  rates,  LAR's  were 
reviewed  and  counted  only  if  they  required  a  drawing 
revision  for  resolution.  These  LAR's  usually  carried  a 
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comment  to  the  effect,  "Information  only,  no  drawing 
revision  required".  This  eliminated  the  LAR's  which  were 
submitted  by  contractors  who  did  not  know  what  they  were 
doing  or  who  were  trying  to  make  up  for  a  low  bid. 
Additionally,  since  a  LAR  is  submitted  for  one  problem  only, 
if  several  drawings  required  revision  as  a  result,  it  was 
counted  as  only  one  revision.  Again,  the  LAR's  are  filed  by 
hull  and  to  get  the  time  series  data,  the  LAR's  had  to  be 
summed  across  all  hulls  in  a  given  month.  The  date  of  a 
revision  was  taken  to  be  the  estimated  drawing  completion 
date  rather  than  the  actual  date  of  the  LAR,  to  establish 
when  it  would  affect  other  ships, 
b.  Reverse  LAR's 

To  reflect  how  many  other  ships  are  affected  by 
a  revision,  the  most  accurate  indicator  is  the  number  of 


reverse  LAR's  issued. 


When  a  LAR  requires  drawing 


revisions,  a  reverse  LAR  is  issued  to  all  ships  whose 
drawing  packages  are  affected,  and  whose  contracts  have 
already  been  awarded.  The  advantage  of  using  reverse  LAR's 
is  that  unaffected  ships  (those  not  receiving  a  given 
SHIPALT  or  who  already  have  the  SHIPALT)  are  not  counted. 
Unfortunately,  the  reverse  LAR  program  was  not  started  until 
1986,  resulting  in  an  incomplete  data  base  for  the  period  of 
the  study.  Therefore  reverse  LAR's  were  not  used. 


lid 
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c.  Number  of  Ships  Affected 

The  next  best  measure  to  capture  the  "snowball" 
effect  of  making  drawing  revisions  is  to  look  at  all  ships 
which  could  be  affected.  This  number  is  actually  an  upper 
limit  on  the  number  of  reverse  LAR's  issued.  Arguably,  the 
FMP  requires  contract  award  by  6  months  prior  to 
availability  start  date,  however,  empirical  observation  of 
actual  award  dates  with  respect  to  availability  start  dates 
showed  4  months  is  closer  to  the  norm.  For  the  purpose  of 
this  study,  number  of  ships  affected  was  defined  as  all 
ships  in  an  availability  or  within  4  months  of  availability 
commencement  for  a  given  month. 

C.  ADJUSTMENTS  TO  GROWTH  DATA  --  '86,  EAST  COAST  DOLLARS 

To  pike  the  cost  figures  commensurable,  two  factors  had 
to  be  considered.  Temporal  differences,  le.,  inflation,  had 
to  be  taken  into  account  as  well  as  regional  differences  in 
labor  rates.  These  factors  could  be  incorporated  using  two 
different  approaches:  1)  by  blocking  the  data  and  using  two 
additional  explanatory  variables  in  the  model  or  2) 
adjusting  the  data  base  for  the  two  effects  prior  to  model 
development.  The  first  approach  was  rejected  since  it  would 
require  a  further  reduction  in  degrees  freedom  in  a  model 
which  already  is  data  limited.  The  second  approach  was 
accomplished  as  follows. 


The  growth  data  were  adjusted  to  1986  dollars. 
Using  Bureau  of  Labor  Statistics  monthly  labor  Indices  for 
the  shipbuilding  industry,  an  Inflation  index  for  1985  and  a 
deflation  index  for  1987  were  derived  by  taking  the 
difference  between  the  average  index  level  for  the  year 
being  adjusted  and  the  average  index  level  for  the  base  year 
and  normalizing  to  the  base  year.  These  indices  were  then 
applied  to  their  respective  groups  of  availabilities.  The 
1985  index  was  1.029  and  the  1987  index  was  0.989. 

2.  Regional  Adjustments 

Due  to  the  wide  geographic  distribution  of 
shipyards,  regional  differences  in  labor  rates  exist. 
Traditionally,  the  shipbuilding  industry  avoids  this 
consideration  by  converting  all  dollar  amounts  to  manhours 
of  labor.  Rather  than  lose  the  magnitude  of  the  growth 
through  a  linear  transformation  to  manhours,  the  growth 
figures  were  simply  transformed  to  constant  location 
dollars,  in  this  case  the  East  Coast.  The  Navy  regularly 
compiles  labor  rates  for  each  contractor,  however  the  data 
is  business  sensitive  and  the  additional  gain  in  accuracy  of 
adjustments  are  not  deemed  significant.  The  U.S.  Department 
of  Transportation's  Maritime  Administration  publishes  an 
annual  report  on  the  Relative  Cost  of  Shipbuilding  which 
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establishes  regional  ratios  relative  to  the  Atlantic  Coast 
(Refs.  2,3].  The  indices  from  that  source  are  in  Table  1. 

Each  year's  regional  indices  were  applied  to  the 
group  of  availabilities  in  each  respective  region.  The 
adjusted  growth  data  and  availability  dates  are  in 
Appendix  C.  The  entire  set  of  time  series  data  are  in 
Appendix  E. 

TABLE  1 

RELATIVE  SHIPBUILDING  COSTS 

PACIFIC  COAST  RATIO 
YEAR  TO  ATLANTIC  COAST 


1985  1.032 

1986  0.998 

1987  1.054" 
*  NAVSEA  estimate 
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A.  FUNCTIONAL  FORM 


In  developing  a  relationship  between  revision  rate  and 
cost  growth  and  attempting  to  capture  the  snowball  effect, 
several  functional  forms  of  the  variables  were  investigated. 
Initially,  only  revision  rate  was  used  as  an  explanatory 
variable.  It  became  apparent  that  revision  rate  by  itself 
did  not  reflect  how  many  ships  were  being  affected  by  each 
revision.  The  progression  was  from  a  simple  linear 

relationship  to  increasingly  complex,  intrinsically 
nonlinear  forms.  The  nonlinear  forms  did  become  linear  upon 
taking  natural  logarithms,  allowing  the  use  of  a  standard 
linear  regression  package, 
l.  Llnaac. Fqehb 

Regressing  growth  on  revision  rate  alone  yielded 
unsatisfactory  results  by  virtually  all  measures  of  model 
performance.  The  next  alternative  was  regressing  growth  on 
the  number  of  ships  affected  which  yielded  the  best  results 
for  a  linear  model,  but  even  these  were  inadequate.  The 
final  alternative  explored  was  a  regression  of  growth  on  the 
revision  rate  and  the  number  of  ships  affected.  This  model 
estimated  a  negative  coefficient  for  revision  rate  which 
intuitively  was  not  correct.  Table  2  lists  the  summary 
statistics  for  the  various  alternatives  and  Figures  1  and  2 
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After  exhausting  the  simple  linear  models,  some 
linear  regressions  with  nonlinear  Interpretations  were 
examined.  Regressions  of  this  form  allow  for  multiplicative 
and  exponential  relationships  among  the  explanatory 


variables 


This  type  of  model  is  intuitively  the  most 


appealing,  since  it  can  capture  the  essence  of  the  snowball 
effect  of  drawing  revisions, 
a.  Log  /  Semi-log 

The  first  of  these  models  attempted  was  obtained 
by  regressing  the  natural  logarithm  of  growth  on  the 
revision  rate,  the  number  of  ships  affected  and  then  both. 
This  class  of  models  was  a  significant  improvement  over  the 
linear  models  in  that  the  coefficients  corresponded  to  the 
best  fit  of  an  exponential  model  in  terms  of  the  explanatory 
variables.  In  other  words,  the  regression  was  of  the  form, 
Ln(Growth)  =  a  +  b(Revision  Rate) 
which  implies  an  underlying  model  of  the  form. 

Growth  ■  EXP  (a  +  b(Revislon  Rate)). 

In  this  form,  the  fit  was  still  not  adequate  and  the 
residuals  were  not  properly  distributed. 

At  this  point,  the  initial  signs  of  upcoming 
problems  also  started  to  appear.  The  model,  with  the  number 
of  ships  affected  as  its  only  variable,  had  a  much  better 
fit  than  the  model  with  only  revision  rate  as  the 
explanatory  variable.  To  confirm  the  apparent  lack  of 
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effect  of  revision  rate,  a  model  with  both  revision  rate  and 
number  of  ships  affected  as  explanatory  variables  yielded  a 
regression  coefficient  for  revision  rate  which  was  not 
significantly  different  from  zero.  Additionally,  the 

Durbin-Watson  statistic  and  the  residual  plots  indicated  a 
high  degree  of  serial  correlation  (see  Table  3).  Draper  and 
Smith  (Ref.  4:pp.  162  -  1691  provide  a  very  concise 

discussion  on  the  derivation  and  significance  of  the 
Durbin-Watson  statistic. 

TABLE  3 

REGRESSION  STATISTICS  FOR  LOG/SEMI -LOG  FORMS 

Variant)  Estimated  Coefficients 
(Significance  Level) 


1)  InG  = 

11.16 

+ 

0 . 15(R) 

(0.000) 

(0.017) 

R-SQ 

=  0.16 

D-W 

=  0.656 

2)  G  - 

8.08 

+ 

0 . 48 (N) 

(0.000) 

(0.000) 

R-SQ 

=  0.48 

D-W 

=  0.419 

3)  G  * 

8.11 

O.Ol(R)  + 

0.48  (N) 

(0.001) 

(0.884) 

(0.001) 

R-SQ 

=  0.46 

D-W 

=  0.423 

G  -  Growth 

R  -  Revision  Rate 

N  -  Number  of  Ships  Affected 

R-SQ  -  R-Squared  Statistic 
D-W  -  Durbin-Watson  Statistic 

Figures  3  and  4  are  representative  of  the  quality  of  fit  of 
this  model  form.  Note  the  poor  distribution  of  residuals 
indicated  by  Figure  4. 


b.  Log  /  Log,  Log  /  Mixed 


This  class  of  models  was  an  attempt  to  do  three 
things:  obtain  a  better  fit,  obtain  more  normally 

distributed  residuals,  and  explore  some  additional  possible 
relationships  of  the  explanatory  variables.  In  the  Log  / 
Log  models  with  the  natural  logarithm  of  growth  regressed  on 
the  natural  logarithm  of  revision  rate  alone,  number  of 
ships  alone,  and  both  together,  the  regression  coefficients 
represent  powers  of  exponentiation  of  the  explanatory 
variables.  The  regression  was  of  the  form, 

Ln(Growth)  =  Ln(a)  +  b  (Ln(Revision  Rate)) 
which  implies  an  underlying  model  of  the  form. 

Growth  =  a  (Revision  Rate)*. 

These  models  provided  a  better  fit,  but  with  the  exception 
of  number  of  ships  alone,  were  not  very  stable  since  the 
fitted  line  was  significantly  influenced  by  a  small  number 
of  outliers.  At  this  point,  the  residuals  appeared  to  be 
normally  distributed  but  still  were  serially  correlated  (see 
Figure  5).  Table  4  shows  the  summary  statistics  for  this 
set  of  alternatives. 

The  final  model  explored  was  a  hybrid  of  the  first 
two  types.  The  natural  logarithm  of  growth  was  regressed  on 
the  natural  logarithm  of  the  number  of  ships  affected  and 


the  raw  revision  rate.  In  this  case,  the  regression  was  of 
the  form, 

Ln(G)  =  a  +  b  (Ln ( N ) )  +  c  (R) 
where  G  Is  the  growth,  N  Is  the  number  of  ships  affected, 

TABLE  4 

REGRESSION  STATISTICS  FOR  LOG/LOG  FORMS 


Variant ) 

Estimated  Coefficients 
(Significance  Level) 

1) 

InG  = 

11.02 

+ 

0 . 77 ( InR ) 

(0.000) 

(0.023) 

R-SQ  = 

0.15 

D-W 

=  0.591 

2) 

InG  = 

5.90 

+ 

3 . 02 ( InN ) 

(0.000) 

(0.000) 

R-SQ  ■ 

0.48 

D-W 

=  0.292 

3) 

InG  = 

5.72 

- 

0 . 29 ( InR ) + 

3.38  (InN) 

(0.000) 

(0.397) 

(0.000) 

R-SQ  = 

0.51 

D-W 

=  0.402 

G  -  Growth 
R  -  Revision  Rate 
N  -  Number  of  Ships  Affected 

R-SQ  -  R-Squared  Statistic 
D-W  -  Durbin-Watson  Statistic 

and  R  is  the  revision  rate.  This  implies  an  underlying 
model  of  the  form, 

G  >  a  (N)»  EXP  (c(R) ) 

This  model  yielded  a  good  fit  and  normal  residuals,  but  the 
coefficient  for  revision  rate  was  still  not  significantly 
different  from  zero  and  the  Durbin-Watson  statistic 
indicated  a  strong  positive  serial  correlation  (see  Table 
5).  Figures  6  and  7  demonstrate  the  fit  obtained  with  this 


model . 


B.  ADDITIONAL  CONSIDERATIONS  -  SERIAL  CORRELATION 

Once  the  various  model  forms  had  been  explored,  the 
problem  of  serial  correlation  remained  to  be  resolved. 
Using  the  hybrid  model  as  the  "best"  (the  most  potential) 
case,  an  autocorrelation  coefficient  was  estimated  using  the 
Durbin-Watson  statistic  and  the  variables  were  then  adjusted 
in  the  manner  described  by  Judge,  et  al.  (Ref.5:pp.  439  - 
444  ]  (see  Appendix  F).  The  regression  on  the  adjusted 
variables  yielded  a  much  better  fit  and  much  better 
residuals  (see  Figures  8  and  9)  but  the  coefficient  for  the 
revision  rate  was  still  not  significantly  different  from 
zero  in  the  adjusted  model  (Table  5). 


TABLE  5 

REGRESSION  STATISTICS  FOR  HYBRID  FORM 
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To  confirm  the  fact  that  revision  rate  had  no 
significant  effect,  the  Log  /Log  model  using  only  the  number 
of  ships  affected  was  adjusted  for  serial  correlation  in  the 
same  manner  described  above  and  a  regression  on  the  adjusted 
variables  performed.  The  results  were  virtually  the  same 
(see  Table  6)  as  the  hybrid  model  indicating  revision  rate 
had  little  effect  on  the  chosen  response  variable,  growth 
defined  using  budget  data. 


TABLE  6 

REGRESSION  STATISTICS  FOR  COMPARISON 


The  original  hypothesis,  that  revision  rate  is  directly 
related  to  cost  growth  during  SHIPALT  installation,  was  only 
weakly  supported  by  the  data  used  in  this  study.  It  is 
believed  that  the  hypothesis  is  sound  and  the  £ault  lies 
with  the  data,  specifically  the  response  variable.  Several 
reasons  exist  for  the  failure  of  the  model  to  support  the 
hypothesis  and  these  will  be  discussed  in  detail. 

A.  REASONS  FOR  MODEL  FAILURE  -  NOISE  IN  THE  RESPONSE 
1.  Estimates  vs.  Actuals 

As  discussed  in  the  chapter  on  database  selection, 
lack  of  complete  data  in  any  central  location  precluded  the 
use  of  actual  return  cost  data  for  model  construction.  By 
defining  growth  as  the  difference  between  the  amount 
programmed  and  the  amount  funded  for  a  given  ship,  the 
growth  figure  is  actually  reflecting  the  accuracy  of 
NAVSEA's  estimates  for  budget  purposes  rather  than  any 
actual  cost  growth  experienced  in  contract  administration. 
While  the  amount  funded  to  a  SUPSHIP  for  a  contract  closely 
reflects  the  contract  award  price,  the  amount  funded 
initially  is  only  90%  of  the  contract  award  price  upon 
contract  award  and  also  includes  long  lead  time  Government 
Furnished  Materials  and  advance  design  work.  Eventually, 
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NAVSBA  has  to  fund  to  at  least  100%  of  the  contract  award 
price  as  well  as  fund  any  growth. 

2.  Long  Run  Approach  to  Estimates 

As  previously  noted,  the  amount  funded  is  not  a 
static  amount,  but  rather  a  dynamic  balance  which  approaches 
the  programmed  amount  as  time  progresses.  Therefore, 
growth,  as  defined,  will  show  an  apparent  decrease  through 
time  for  a  given  ship.  This  effect  is  evident  in  Figure  10, 
as  can  be  seen  in  a  comparison  of  the  aggregate  growth  for 
the  FFG-7  class  in  1985  and  1986.  The  1985  growth  data  are 
consistently  lower  than  the  1986  data,  even  after  adjusting 
for  inflation.  This  is  presumably  due  to  the  longer  period 
the  1985  data  have  had  to  approach  the  programmed  amount. 

3.  Effect  of  Learning  .on  Eat  Irate.  .Accuracy 

Another  trend  which  can  be  seen  in  the  data  is  the 
effect  of  NAVSEA's  continuous  review  process  of  their  budget 
estimates.  Comparing  the  1986  and  1987  data  shows  the 
apparent  growth  for  1987  decreasing.  This  is  most  likely 
due  to  NAVSEA's  revising  their  budget  estimates  downward 
following  a  review  of  their  1986  program  amounts  for  the 
FFG-7  class.  This  change  in  estimate  procedures  can  cause 
the  apparent  growth  to  either  increase  or  decrease. 

4.  Sensitivity  to  Bidding  Practice  of  Contractors 

NAVSEA's  practice  of  funding  to  90%  of  the  contract 

award  price  makes  this  data  set  extremely  vulnerable  to  a 
low  bid  by  a  contractor.  An  artificially  low  award  price 
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Figure  10.  Growth  vs.  Time 
With  and  Without  the  FFG-10 


will  result  In  a  low  funding  level  and  a  disproportionately 
large  amount  of  apparent  growth  on  that  specific  contract. 
A  prime  example  of  this  problem  is  the  PFG-10's  effect  on 
monthly  cost  growth  data  for  the  period  of  its  contract, 
9/85  -  12/85.  A  major  share  of  the  growth  attributed  to 
those  months  is  comprised  of  the  one  contract  on  the  FFG-10. 
This  can  be  seen  in  Figure  10,  which  is  a  plot  of  Growth  vs. 
Time  with  the  FFG-10  in  the  data  base  and  the  FFG-10  out  of 
the  data  base. 

B.  INSIGHTS  GAINED 
1.  Revision  Rate 

The  revision  rate,  as  measured  at  the  SPY,  seems  to 
be  inversely  related  to  EPY  product  quality.  Although  this 
study  provides  no  conclusive  relationship  between  EPY 
product  quality  and  cost  growth,  this  is  most  likely  due  to 
the  choice  of  the  response  variable  and  the  fact  that  any 
influence  of  the  revision  rate  on  growth  is  lost  among  the 
other  contributing  factors  to  "growth"  variability. 

Initially,  revision  rate  was  thought  to  capture  the 
"snowball"  effect  as  well  as  the  product  quality.  Upon 


further  analysis,  this  did  not  appear  to  be  the  case.  An 
additional  explanatory  variable  is  needed  to  establish  this 
important  characteristic. 


To  establish  the  fact  that  more  ships  are  affected 
by  a  drawing  revision  than  merely  the  ship  initiating  the 
LAR,  one  of  two  potential  variables  can  be  included  in  the 


mode  1 . 

a.  Number  of  Ships  Potentially  Affected 

As  discussed  in  the  database  development 
chapter,  this  was  taken  to  be  all  ships  in  an  availability 
or  within  4  months  of  their  availability  start  date.  This 
number  could  just  as  easily  have  been  all  ships  in  an 
availability  or  all  ships  in  an  availability  or  within  6 
months  of  their  availability  start  date,  whichever  scenario 
fits  the  situation.  The  point  to  remember  is  that  this 
number  is  an  upper  limit  since  not  all  ships  with  contracts 
already  awarded  are  necessarily  affected  by  a  particular 
drawing  revision. 

b.  Number  of  Reverse  LAR's  Issued 

A  more  accurate  measure  of  the  number  of  ships 
affected  by  a  specific  drawing  revision  is  the  number  of 
reverse  LAR's  issued  by  the  SPY  since  reverse  LAR's  are  only 
issued  to  affected  ships.  This  measure  was  not  used  since 
the  procedure  was  not  Implemented  until  1986. 

3.  Serial  Correlation 


The  strong  positive  serial  correlation  evidenced  by 
the  Durbin-Watson  statistics  and  the  residual  plots  of  all 

•a 

model  forms  was  a  significant  but  not  surprising  result 
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since  each  availability  covers  several  time  periods.  By 
estimating  the  coefficient  of  serial  correlation  and 
adjusting  the  variables  accordingly,  the  fit  of  the  model 
can  be  greatly  enhanced. 

C.  POTENTIAL  APPLICATION  OF  THE  METHODOLOGY 

Should  this  method  produce  satisfactory  results  in 
further  work  with  a  proper  response  variable,  it  can 
potentially  be  used  for  evaluating  the  impact  of  the  quality 
of  planning  yard  documents  on  installation  cost  growth  for 
any  large  class  of  ships  with  a  short  availability  cycle. 
The  basic  assumptions  are  simply  that  the  availabilities  are 
of  short  enough  duration  to  warrant  averaging  the  growth 
over  the  period  of  the  availability  and  that  all  contracts 
are  of  generally  the  same  type,  e.g.,  fixed  price. 


1 

V.  RECOMMENDATIONS 

A.  FUTURE  AREAS  TO  EXPLORE 

1.  AUTOMATED  ACCOUNTING  DATA  --STAR 


With  the  SUPSHIP's  online  with  STAR,  the  ability  to 
obtain  actual  expenditure  data  instead  of  estimates  will 
likely  be  the  single  most  significant  factor  in  determining 
the  extent  to  which  revision  rate,  as  a  measure  of  EPY 
product  quality,  affects  cost  growth.  This  will  allow  the 
calculation  of  actual  cost  growth  on  a  given  contract. 
Additional  information  needed  will  be  the  SUPSHIP  estimate 
of  NAVSEA  and  the  TYCOM  shares  of  both  the  award  price  and 
the  final  price. 

2.  Revet as  LAR's 

Using  reverse  LAR's  as  the  measure  of  how  many  ships 
are  affected  by  a  revision  will  accomplish  two  things. 
First,  ships  not  affected  will  not  be  counted.  Second,  the 
arbitrary  A-minus  date  is  no  longer  a  factor  to  be 
considered  since  it  is  already  considered  in  determining 
which  ships  receive  reverse  LAR's. 

3.  Verification 

In  the  case  of  the  FFG-7  Class  and  its  EPY,  Long 
Beach  Naval  Shipyard,  the  SUPSHIP's  are  currently  providing 
informal  feedback  in  the  form  of  estimated  cost  growth 
attributable  to  a  specific  revision.  Since  this  is  exactly 
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the  relationship  this  study  is  attempting  to  establish,  any 
future  efforts  using  this  methodology  will  have  an 
alternative  source  against  which  to  verify  the  results. 

B .  MODBL  FORM 

The  most  promising  functional  form  of  the  model  is  the 
hybrid  model  with  the  number  of  ships  affected  as  a 
multiplicative  form  and  the  revision  rate  as  an  exponential 
form.  This  model  seems  to  have  the  best  distribution  of 
residuals  and  has  a  good  representation  of  the 
interrelationship  of  the  two  proposed  factors  which 
influence  cost  growth.  Adjusting  the  data  for  serial 
correlation  will  significantly  reduce  the  uncertainty  in  the 
fitted  model  for  whatever  form  is  chosen,  and  will  also 
allow  more  accurate  forecasts  of  the  EPY’s  Impact  on  the 
installing  activities. 
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glossary 


Availability:  Overhaul  period  for  a  given  ship. 

Availability  Cycle:  The  time  between  availabilities  for  a 

given  ship. 

Drawing  Schedule:  List  of  all  applicable  drawings  for  a 
given  ship  and  a  giv?n  SHIPALT. 

Expanded  Planning  ¥ard  (EPY ) :  The  planning  /  design 

activity  for  SHIPALT  development. 

Fleet  Modernization  Program  ( FMP ) :  The  overall  program  for 
prioritizing,  programming,  planning,  and  implementing 
improvements  to  existing  ships. 

Funding:  The  obligation  of  money  to  various  accounts. 

Liaison  Action  Record  (LAR):  A  request  from  an  installation 
activity  to  the  EPY  for  design  assistance.  Resolution 
requires  either:  1)  clarification  of  existing  drawings  or, 
2)  revision,  addition,  deletion  of  existing  drawings. 

Naval  Sea  Systems  Command  (NAVSBA) :  The  Chi_f  of  Naval 

Operation's  (CNO)  agent  for  execution  of  the  FMP. 

Program:  1)  In  a  planning  sense,  the  assignment  of  a  set  of 

SHIPALT's  to  be  accomplished  on  a  given  ship  during  a 
specified  availability. 

2)  In  a  fiscal  sense,  the  assignment  of  money  to 
different  accounts. 

Ship  Alteration  (SHIPALT):  A  design  package  to  update  an 

existing  system  on,  add  a  new  system  to,  or  remove  an  old 
system  from  a  given  ship  class. 

Ship  Class:  All  ships  sharing  a  common  design,  e.g.,  the 

FFG-7  class. 

Supervisor  of  Shipbuilding,  Conversion,  and  Repair 
(SUPSHIP):  The  contracting  activity  for  installation  of 

SHIPALT's  and  overhaul  of  ships  in  the  private  sector. 
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Raw  Growth  Data  bv  Hull 

Fiscal  Program  data  as  recorded  in  the  SAFIRE  (budget 
execution  subsystem)  data  base  as  of  December,  1987.  Raw, 
unadjusted  data  with  growth  as  defined  for  this  study. 

Growth  =  Curr.  Prog.  -  Tot.  Funded. 


East  Coast  FY85  availabilities: 


Hull 

Current 

Funded 

+  Prior 

Funded 

=  Total 

Funded 

Current 
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11 
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13 
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113849 
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22 
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128891 
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26 

603270 
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28 

605746 
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715208 

725766 

10588 

29 

749760 

116330 

866090 

878461 

12371 

32 

627508 

100000 

727508 

749298 

21790 

34 

1045164 

100000 

1145164 

2034509 

889345 

East 

Coast  FY86 

availabilities: 

Current 

+  Prior 

■  Total 

Current 

Growth 

Hull 

wmim-Tzm 

■■KZTTTnnHi 

■1  Hill 

7 

2506496 

413345 

2919841 

3686566 
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15 

2075863 

434438 

2510301 
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168061 

21 

2263729 
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26 

1178305 

168550 

1346855 
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31 
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81579 

36 
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39 
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54934 

42 

100000 

0 
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594961 
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+  Prior 

=  Total 
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Growth 
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20 
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24 
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40 
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45 
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Current  +  Prior  =  Total 


4486575 
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70000 
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1394586 
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West  Coast  PY86  availabilities: 

Current  +  Prior  =  Total 
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3376555 
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1454164 
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3060 
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Current  +  Prior  « 
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Current 
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125879 


Growth 


110339 

627250 

347733 

557147 

1115714 

0 


Growth 
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Adjusted  Growth  Data  bv  Hull 

Growth  data  adjusted  to  1986,  East  Coast  Dollars 
Ordered  by  hull  number. 


START  STOP 

HULL _ GROWTH _ GA1E _ DATE 
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9 
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15 
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24 
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26 
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32 
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34 
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50927 
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36 

32501 

60213 

60512 

37 

0 

60224 

60620 

38 

92902 

70202 

70403 

39 

54934 

60213 

60506 

40 

59031 

70105 

70306 

41 

2138 

70105 

70213 

42 

494961 

61013 

61212 

43 

3490 

70316 

70508 

45 

57323 

70203 

70410 

Growth  data  adjusted  to  1986,  East  Coast  Dollars. 
Ordered  on  availability  start  date. 
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Growth  per  month  by  hull,  calculated  by  computing  daily 
average  and  multiplying  by  the  number  of  days  that  hull  was 
in  an  availability  in  a  given  month.  Totalling  each  column 
yields  the  time  series  growth  data  used  for  the  analysis. 
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APPENDIX  E 


Time  Series  Variables 


MO/YR 

GROWTH 

REVISIONS 

#  SHIPS 

1/85 

13301 

0 

7 

2/85 

21872 

5 

8 

3/85 

26961 

8 

9 

4/85 

56075 

10 

8 

5/85 

45127 

4 

8 

6/85 

64529 

2 

6 

7/85 

144651 

6 

7 

8/85 

122149 

5 

8 

9/85 

369226 

12 

9 

10/85 

1101256 

3 

11 

11/85 

1087780 

7 

11 

12/85 

881381 

11 

12 

1/86 

211775 

12 

10 

2/86 

237960 

10 

10 

3/86 

262954 

10 

10 

4/86 

358991 

13 

11 

5/86 

371406 

9 

9 

6/86 

410020 

5 

8 

7/86 

818026 

4 

8 

8/86 

733305 

6 

7 

9/86 

799404 

13 

9 

10/86 

730046 

2 

11 

11/86 

834086 

9 

11 

12/86 

682416 

15 

10 

1/87 

619851 

5 

9 

2/87 

540956 

12 

9 

3/87 

329413 

1 

6 

4/87 

27083 

1 

4 

5/87 

3667 

1 

2 
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APPENDIX  P 

Autocorrelation  Adjustment 

In  ordinary  least  squares  estimation  procedures,  it  is 
assumed  that  the  residuals  have  a  constant  variance  and  zero 
mean.  The  regression  coefficients,  0,  are  estimated  using  a 
model  of  the  form, 

y  =  X0  +  e 

The  first  order  autocorrelation  indicated  by  the 
Durbin-Watson  statistics  is  a  special  case  of  the  above 
model  where  the  constant  variance  assumption  is  violated. 
To  regain  a  constant  variance,  the  model  is  modified  by 
assuming  the  residuals  are  stochastic  quantities  partially 
determined  by  previous  observations.  In  this  case, 

et  ■  r  *t-i  *  vt 


and  v  has  constant  variance  with  zero  mean. 

To  accomplish  this  transformation,  a  coefficient  of 
serial  correlation  is  estimated  using  the  Durbin-Watson 
statistic  from  the  unadjusted  regression  model, 

t  =  1  -  (D.W.  /  2) 


The  variables  are  then  adjusted  in  the  following  manner, 
for  t  *  2,T 


for  t  =  1 , 


*t 
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tx 


t-1 
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After  adjusting  the  variables,  the  regression  coefficients 
are  estimated  using  least  squares  on  the  transformed  model 

*  *  * 
y  =  X  0  +  e 

The  following  table  contains  the  raw  and  adjusted  data  for 
the  hybrid  model  form  discussed  in  Chapter  3. 


G  --  GROWTH 

»  --  NUMBER  OF  SHIPS  AFFECTED 
Rev  --  REVISION  RATE 


Unadjusted  Data 


Adjusted  Data 


1/85 

9.50 

1.95 

0.00 

5.085 

1.042 

0.000 

2/85 

9.99 

2.08 

5.00 

1.974 

0.436 

5.000 

3/85 

10.20 

2.20 

8.00 

1.763 

0.441 

3.778 

4/85 

10.93 

2.08 

10.00 

2.319 

0.224 

3.244 

5/85 

10.72 

2.08 

4.00 

1.483 

0.323 

-4.445 

6/85 

11.07 

1.79 

2.00 

2.024 

0.036 

-1.378 

7/85 

11.88 

1.95 

6.00 

2.529 

0.433 

4.311 

8/85 

11.71 

2.08 

5.00 

1.679 

0.436 

-0.067 

9/85 

12.82 

2.20 

12.00 

2.928 

0.441 

7.778 

10/85 

13.91 

2.40 

3.00 

3.086 

0.542 

-7.134 

11/85 

13.90 

2.40 

7.00 

2.151 

0.373 

4.467 

12/85 

13.69 

2.48 

11.00 

1.951 

0.460 

5.089 

1/86 

12.26 

2.30 

12.00 

0.703 

0.204 

2.710 

2/86 

12.38 

2.30 

10.00 

2.024 

0.358 

-0.134 

3/86 

12.48 

2.30 

10.00 

2.025 

0.358 

1.555 

4/86 

12.79 

2.40 

13.00 

2.252 

0.453 

4.555 

5/86 

12.83 

2.20 

9.00 

2.023 

0.172 

-1.979 

6/86 

12.92 

2.08 

5.00 

2.093 

0.224 

-2.601 

7/86 

13.61 

2.08 

4.00 

2.700 

0.323 

-0.223 

8/86 

13.51 

1.95 

6.00 

2.008 

0.190 

2.622 

9/86 

13.59 

2.20 

13.00 

2.186 

0.554 

7.933 

10/86 

13.50 

2.40 

2.00 

2.023 

0.542 

-8.979 

11/86 

13.63 

2.40 

9.00 

2.233 

0.373 

7.311 

12/86 

13.43 

2.30 

15.00 

1.919 

0.278 

7.400 

1/87 

13.34 

2.20 

5.00 

1.993 

0.253 

-7.668 

2/87 

13.20 

2.20 

12.00 

1.938 

0.342 

7.778 

3/87 

12.71 

1.79 

1.00 

1.557 

-0.064 

-9.134 

4/87 

10.21 

1.39 

1.00 

-0.523 

-0.127 

0.155 

5/87 

8.21 

0.69 

1.00 

-0.412 

-0.478 

0.155 
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